Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Using project_id output is not forcing to wait for the project creation #601

Conversation

lsierant
Copy link

This fixes #598.

Using project_id output with this fix will force terraform to wait on project creation and resolve dependencies properly. This is primarily fix for a case when there are datasources that are gathering any data using project_id. These datasources are run immediately and fail in plan phase, because project_id value output was available immediately, not after creating the project.

One warning to consider though. After this change project_id output value will not be available until after apply. This causes terraform to error if project_id output is used anywhere in for_each clauses. Terraform cannot handle resources without knowing upfront how many resources there will be. To avoid that, it is needed to perform terraform apply -target module.project to create a project first, then second standard apply to create resources that are depending on it.

I haven't run any tests, because I don't know whether there is any existing test env or every contributor should test on their own gcp org.

@lsierant lsierant requested a review from a team June 30, 2021 11:30
@comment-bot-dev
Copy link

Thanks for the PR! 🚀
✅ Lint checks have passed.

Copy link
Member

@bharathkkb bharathkkb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @lsierant

@@ -16,7 +16,7 @@

output "project_id" {
description = "Project id (depends on services)."
value = google_project.project.project_id
value = trimprefix(google_project.project.id, "projects/")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't depends_on = [google_project. project] work?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure which depends_on you're asking:

  1. if you're asking about depends_on here in output then no, changing
  depends_on  = [google_project_service.project_services]

to

  depends_on  = [google_project.project]

does not work. Terraform ignores this explicit dependency when it has the output value available immediately. This is especially the case when output value comes from input attribute.

  1. If you're asking whether explicit depends on module.project works e.g. in a datasource that is using this output, then yes, it's working. But please see my comment in main thread.
data "google_storage_project_service_account" "gcs_account" {
  project = module.project.project_id
  depends_on = [module.project]
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, I assumed depends_on may block on google_project.project since some of the attribs are computed.

@bharathkkb
Copy link
Member

Also

these datasources are run immediately and fail in plan phase

Could you show an example. Usually you can defer datasources to apply time using depends_on within datasources.

@lsierant
Copy link
Author

lsierant commented Jul 1, 2021

Also

these datasources are run immediately and fail in plan phase

Could you show an example. Usually you can defer datasources to apply time using depends_on within datasources.

Example code:

terraform {
  required_version = ">= 0.13"

  required_providers {
    google      = ">= 3.71.0"
  }
}

provider "google" {
  region = "europe-central2"
}

module "project" {
  source  = "terraform-google-modules/project-factory/google//modules/fabric-project"
  version = "11.0.0"

  name            = "test-project-name-43413"
  billing_account = "<BILLING_ACCOUNT_ID>"
  parent          = "folders/<FOLDER_ID>"
  prefix          = "dev"

  auto_create_network = false

  activate_apis = [
    "compute.googleapis.com",
  ]
}

data "google_storage_project_service_account" "gcs_account" {
  project = module.project.project_id
}

output "project-id" {
  value = module.project.project_id
}

That causes datasource to fail:

$ terraform plan

 Error: Error when reading or editing GCS service account not found: googleapi: Error 400: Unknown project id: 'dev-test-project-name-43413', invalid

   with data.google_storage_project_service_account.gcs_account,
   on main.tf line 29, in data "google_storage_project_service_account" "gcs_account":
   29: data "google_storage_project_service_account" "gcs_account" {

Of course explicitly depending on project solves this issue, but it's counter-intuitive.

data "google_storage_project_service_account" "gcs_account" {
  project = module.project.project_id
  depends_on = [module.project]
}

Especially if you look into project_id output in modules/fabric-project/outputs.tf, where there is explicit depends_on not even on project but on project services. Which makes sense, because you don't want to use project_id if it doesn't have APIs enabled.

output "project_id" {
  description = "Project id (depends on services)."
  value       = google_project.project.project_id
  depends_on  = [google_project_service.project_services]
}

But this depends_on in output does not work, because project_id is available immediately. I'm not sure if this is not a bug in terraform, anyway current behaviour is surprising. project_id is often used in many resources and specifying explicitly dependencies on module.project is possible, but tedious.

@@ -16,7 +16,7 @@

output "project_id" {
description = "Project id (depends on services)."
value = google_project.project.project_id
value = trimprefix(google_project.project.id, "projects/")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, I assumed depends_on may block on google_project.project since some of the attribs are computed.

@bharathkkb bharathkkb merged commit 551833a into terraform-google-modules:master Jul 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Using project_id output is not forcing to wait for the project creation
3 participants